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Introduction 

The Department of Defense (DoD) has an annual budget approaching a half trillion 
dollars. A significant portion of that budget is either directly or indirectly affected by 
Information Technology (IT) Infrastructure modernization initiatives. Nationally, investment in 
IT infrastructure modernization is about $250 Billion a year, spanning approximately 175,000 
projects. Unfortunately, various studies indicate that only 28% are completed on time and on 
budget—with the number dropping to around 9% for larger companies in 1994, including 
government programs. By 1998, these numbers had improved, with success rates for larger 
companies, for example, up to 24%. But, only small organizations have managed to 
implement more than half of their applications into one integrated system (Smith, 2000; 
Johnson, 1999, December; Keller, 2006, May 29). 

The factors driving these numbers are myriad—and include the sheer scope and 
complexity of infrastructure modernization programs, unstable/undefined requirements, 
unstable funding, moving target objectives, and evolving threats. As a result, the costs of 
integration complexity increase exponentially, but yet are almost invariably under estimated. 
These challenges notwithstanding, one Gartner study asserts that IT asset productivity will 
drive market capitalization (Gartner, 2002, July). 

Given the scope, importance and complexity of these projects, reliable, cost- 
effective, early warning indicators of problems are essential. Yet, classical investment theory 
provides little guidance for dealing with public-sector investment. The result is a general 
absence of computationally efficient, predictive models applicable to the analysis of those 
investments. 

However, there is progress on several fronts. At the micro level, Earned Value 
Management is gaining acceptance. But, more comprehensive, flexible methods can be 
developed by viewing a Firm as an engineered artifact whose responses to a range of inputs 
can be characterized in terms of duration, mass, time, stability, and location. (By way of 
terminology, a “Firm” refers to both public and private organizations with investment 
responsibilities.) 

How well a Firm executes an investment depends largely on how well it acquires and 
uses information. The efficiency of that use provides a basis for pricing the value of a Firm, 
independent of valuations derived from equity markets. 
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For the private sector, Capital Asset Pricing Models (CAPM) models provide 
valuation tools, of which the Black-Scholes equation is the most well known and successful 
example. These models are based on the observation that equity markets have properties 
that can be analyzed using models based on the “Law of Large Numbers.” 

Unfortunately, the conditions enabling the quantitative analysis of private-sector 
investments do not apply in the public sector because there is no market for public-sector 
goods. Nonetheless, all Firms, regardless of whether they are in the private or public sector, 
must respond to a range of random (external and internal) perturbations. The responses can 
be accurately modeled using mathematical processes based on the “Law of Large 
Numbers.” In particular, the operation of a Firm can be modeled as a stochastic feedback 
system using algorithms from Information Theory and System Control Theory. 

IT Infrastructure Investments—in the Public Sector 

Private equity markets provide a profit incentive to resolve uncertainty, which has as 
one of its effects the rapid aggregation of information from large numbers of participants. In 
general, the larger and more diverse the number of sources, the more accurate is a Firm’s 
valuation. (With the Web and the Internet dramatically lowering the cost of information, 
similar aggregation effects are occurring in the news media, entertainment, and politics, as 
evidenced by the impact of Napster, U-Tube, Google, etc.) 

Unfortunately, the Public Sector lacks an incentive mechanism and usually consists 
of a few service/product providers, and generally only one customer—the government. 
(Markets with few sellers and buyers are more suited to analysis by Game Theory.) 

Despite these differences, private and public sectors share a range of common 
concerns, especially for large scale IT modernization projects. Among these is a compelling 
need to answer questions such as: 

■ What level of uncertainty surrounds cost/schedule estimates, especially at the onset 
of a major investment in IT infra-structure? 

■ Under what conditions will risk/uncertainty decline or rise? And at what rate? 

■ How can risk/uncertainty concerning cost, schedule, and scope be identified in a 
timely manner, quantified, and mitigated? 

■ How can a Firm best respond to disruptions to supply, budget and schedule or to the 
introduction of new technologies by competitors? 

■ At what point in a project-development cycle will estimates of cost, schedule, and 
scope become both stable and credible? 

■ How can confidence levels in cost and schedule estimates be measured? 

■ Are project requirements under active management sufficiently stable to ensure 
project completion on time and within budget? 

■ What (quantitative) models can be used to determine the maximum effective rate of 
investment for public-sector projects? 

■ How can an optimal investment portfolio be constructed? 

■ And, how is such a portfolio optimality measured? 
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■ How do the performance measures such as requirements stability, work package 
completion, rework rates and resolution rates for major and minor issues correlate 
with each other? 

■ Is there a level of disruption or tipping point to the schedule, cost, and resource 
allocations beyond which a project cannot recover? 

Developing a quantitative framework capable of answering these questions is the 
objective of the paper. The strategy is to employ key parameters governing the efficiency of 
a Firm’s operation in terms of observable, easy-to-compute variables in mathematical 
models of the underlying process dynamics. 

Portfolio Investment Management—An Overview 

Large-scale Defense infrastructure modernization programs such as Global Combat 
Support have complex inter-dependencies and long-time horizons that render fully informed 
investment decisions difficult to achieve before substantial, and unrecoverable, resources 
are committed. However complex these decisions, they, nonetheless, can be decomposed 
along three basic dimensions: 

■ Uncertainty 

■ Timing 

■ Irreversibility 

These primary parameters define the value of investment options available to a Firm, 
regardless of whether it is in the public or private sector. Unfortunately, algorithms capable 
of modeling the effects of these variables are relatively few, especially for the uncertainty 
and irreversibility of investment decisions (Dixit & Pyndik, 1994, p. 211). For large-scale 
Information Technology (IT) modernization programs, there are at least three sources of 
uncertainty—and, thus, risk 

■ The technical complexity 

■ The programmatic complexity of integrating software intensive systems 

■ The absence of accurate cost information at the onset of major systems/ software 
programs 

Software-intensive systems are particularly sensitive to the systematic under¬ 
estimation of risk, primarily because the level of complexity is hard to manage, let alone 
comprehend. Investment in software-intensive systems tends to be irreversible because it is 
spent on the labor required to develop the intellectual capital embedded in software. 

The outcome of software development is almost invariably unique, a one-of-a-kind 
artifact—despite the numerous efforts to develop reusable software. Unlike physical assets, 
the salvage value of software is zero because no benefit is realized until the system is 
deployed; and that labor, once invested, is unrecoverable. One result is an (implicit) 
incentive to continue projects that have little chance of success, despite significant cost 
overruns, schedule delays. 

Indeed, an analysis of several hundred NASA projects indicates that accurate 
estimates at project onset are virtually impossible to achieve, which raises concern for the 
validity of initial Planned Value estimates since they are the basis of Earned Value 
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calculations (Suter, 2005, p. 261). Thus, measures of uncertainty for cost/schedule 
estimates and the rate at which that uncertainty declines are a key concern—because, they 
govern whether and to what extent confidence can be placed in cost and schedule 
estimates. The key to overcoming initial estimate uncertainty is the capability to harness and 
to apply information as it becomes available, thus, enabling a Firm to capture the time value 
of that information. 

Indeed, where IT infrastructure modernization projects are supported by a strong 
quality-assurance, systems-engineering culture (e.g., have institutionalized best-practice 
regimes such as the CMMI, 6-Sigma, Agile Methods) are likely to quickly reduce estimate 
errors incurred at project start-up. Firms without that culture tend to have limited information 
efficiency. (Drawing an analogy to thermo-dynamic systems, such Firms constitute highly 
dissipative systems in that they exhibit a high degree of entropy, which takes the form of 
information disorganization). 

Unfortunately, traditional methods of discounting investment risk such as Net Present 
Value (NPV) do not account for irreversibility and uncertainty. In part, this is due to the fact 
that NPV computes the value of a portfolio of investments as the maximized mean of 
discounted cash flows on the assumption that the risk to underlying investment options can 
be replicated by assets in a financial market. 

NPV also implicitly assumes that the value of the underlying asset is known and 
accurate at the time the investment decision is made. 

These assumptions seldom apply for large-scale infra-modernization programs, in 
either the public or the private sector. In addition, NPV investment is undertaken when the 
value of a unit of capital is at least as large as its purchase and installation costs. But, this 
can be error prone since opportunity costs are highly sensitive to the uncertainty 
surrounding the future value of the project due to factors such as the riskiness of future cash 
flows. These considerations also extend to econometric models, which exclude 
irreversibility, the incorporation of which transforms investment models into non-linear 
equations (Dixit & Pindyck, 1994, p. 421). Nonetheless, irreversibility constitutes both a 
negative opportunity cost and a lost-option value that must be included in the cost of 
investment. 

In addition, the competitive equilibrium of a market is virtually never stationary, even 
in the long run. Rather, it is a dynamic process in which prices can fluctuate widely, and, 
thus, contribute to uncertainty. Neither classical investment theory nor discounting methods 
such as NPV take these factors into account. Yet, for long-term, capital-intensive 
investments such as oil exploration and IT infrastructure modernization, price fluctuation 
constitutes significant risk—which must be factored into investment decisions (1994, p. 396). 

These difficulties are due to an underlying limitation common to both classical 
investment theory and valuation methods such as NPV: their reliance on simple equilibrium 
relationships between rates of investment and risk, which has the practical effect of 
precluding the effect of uncertainty and irreversibility on investment. The factors that 
adversely impact NPV also impact the accuracy of Planned Value benchmarks that are the 
basis of Earned Value Calculations (Suter, 2006, p. 406). For these reasons, ’’classical” 
methods have met with little success in providing accurate valuations of Public investments 
and qualified success for those in the Private Sector. 
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Capital Asset Pricing models such as Black-Scholes, however, sidestep these 
problems by transforming the analysis from a deterministic formulation to one based on 
probability. To handle uncertainty and risk driven by price fluctuations, it uses the Ito Lemma 
to compute valuations. Black-Scholes proceeds from the assumption that there is a true 
value for a stock that corresponds to its risk, and that value can be used to decide whether 
the market price for a stock is too high or too low. That is, an option’s value equals the value 
of the information concerning that risk. 

Black-Scholes models the price of a stock option as a Market-driven process defined 
by Eqn [3.1], the fundamental condition of equilibrium (Dixit & Pindyck, 1994, p. 115; Cover 
& Thomas, 1991, p. 28): 

[3.1] p = r + 9 *ct * Pxm 

Where: 

p = the risk adjusted return 

0 = market price of risk 

r = riskless rate of return 

Pxm = coefficient of correlation between returns on the particular asset (“x” subscript) 
and the entire market portfolio of stocks (denoted by the “m” subscript). 

a = the proportional variance parameter 

dz = the increment of the standard Wiener process 

The computational efficiency of Black-Scholes model enables Floor Traders (and 
computer-programmed trading algorithms) to exploit small, short-term price fluctuations in 
real time and to use new pricing information to continually rebalance portfolios. Investors (as 
distinguished from Floor Traders by virtue of their longer-term time horizon) find Black- 
Scholes no less useful because of its ability to link risk to valuation over timeframes ranging 
into years. More recent refinements such as Levy processes introduce more realism by 
generalizing Brownian motion processes to include discrete state jumps. (The jumps can be 
local, global, simultaneous, independent or correlated.) 

The significance of Black-Scholes is its computational efficiency for modeling price, 
interest, and discount rates—using a few readily observable parameters that provide 
reasonable approximations to the underlying physical processes using methods based on 
the Law of Large Numbers. In particular, it specifically eschews unobservable/hard-to- 
measure parameters such as “investor psychology.” The Black-Scholes strategy is at 
variance with efforts to improve the analysis by increasing the dimensionality of the problem 
via the addition of more parameters (as is often done with econometric models, and on 
occasion with Balance Scorecard methods). The addition of extra parameters may well 
provide a more detailed picture of performance, but at additional cost and without 
necessarily improving the accuracy needed for decision-making (McShea, 2006, November, 
p. 31). As a result, these refinements have added relatively minimal value and can be 
relatively expensive to implement. 
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The key is to recognize that decision-making is not necessarily dependent on a 
detailed understanding of causality to be effective. Thus, Capital Asset Pricing models focus 
on those few variables with the most explanatory power. The objective is not to predict 
which firms are most likely to succeed, but only what they are “worth” as measured against 
various combinations of risk, uncertainty, interest rates, and competing investment 
opportunities. The task of this paper is to provide equivalent, computationally efficient 
methods for estimating valuations in public-sector investments, using the fewest parameters 
with the most predictive power. 

There are, of course, methods other than market-based Capital Asset Pricing for 
determining asset valuations. But, these also suffer from various limitations. For example, 
Dynamic Programming (DP) could be applied to public-sector investments and is useful in 
solving multi-stage optimization problems, but only if a small number of possible choices 
exist at each stage. Indeed, a small increase in the number of possible choices leads to a 
combinatorial explosion, thus curtailing overall efficiency of DP. 

Another method is the Discrete Binomial Model, which uses a risk-adjusted 
stochastic process for modeling the underlying asset. The strategy is to approximate 
uncorrelated investment dynamics using two basic ideas: a change of time scale and a 
change of the basis of the asset span to approximate uncorrelated geometric Brownian 
motion. Yet another variant, the Lattice Binomial model, has proved useful for valuing 
complex option problems when payoff depends on multiple state variables that follow 
correlated geometric Brownian processes. In this case, the strategy is to approximate a 
multi-dimensional geometric Brownian motion with a binomial lattice by choosing the size 
and the probabilities of the jumps so that the characteristic function of the discrete 
distribution converges to the characteristic function of the continuous distribution. 

But, both methods require knowledge of the underlying probability distributions—a 
requirement that can be difficult to satisfy—and will converge to a solution only in the limit. 
For practical applications, the time required to acquire sufficient data to identify a convergent 
solution can preclude widespread application, especially if the time value of information 
rapidly declines, thus forcing the decision-maker to decide on acting with incomplete 
information, or on risking being overtaken by events. 

Information Theory and “Synthetic Prices” 

In competitive markets, a single number—the price of a Firm’s stock—represents 
risk. Under ideal conditions that price fully captures the Firm’s internal efficiencies, Return- 
on-lnvestment (ROI) and earning potential. Those efficiencies determine the Firm’s 
capability to harness new information as it becomes available. For private-sector firms, 
prices provide two important types of information: 

1. The rate at which information becomes known. 

The rate is analogous to the diffusion problem in heat transfer, which means that 
information diffusion can be modeled as a Brownian motion processes. 

2. How information is aggregated. 

In the private sector, the efficiency of aggregation indicates market efficiency; yet, 
while no such aggregation occurs in public-sector markets because there is no incentive. 
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Hayek was the first to identify these effects and to provide the rationale for defining 
markets in terms of their information value. The definition, in highly abbreviated form, is as 
follows: 


Competitive markets provide for the efficient coordination of decisions involving time 
and uncertainty. The process can be modeled as a Random Walk (in which the limit 
is approximated by a Brownian motion process). The rationale derives from the fact 
that where information flows without impediments, stock prices immediately reflect 
the latest information—so that a price change today will reflect only today’s news and 
is independent of any prior price change (such processes are a weak form of market 
efficiency and have been modeled with some success using Markov models) (Dixit & 
Pindyck, 1994, p. 63). And, since news often is unpredictable, price changes are also 
unpredictable, but fully reflect all known information—thus, justifying the Random 
Walk interpretation. 

However, competitive markets are not the only mechanism for determining 
investment valuations. A Firm’s internal (information) processing efficiencies enable it to 
reallocate resources based on new information—and, thus, to manage risk. The efficiency of 
that process can be measured independently of market-based valuation. In this way, it can 
be applied to public-sector Firms to construct “synthetic prices”—thus linking valuations of its 
investments to its asset of a Firm: the ability to process information efficiently. 

For the private sector, perturbation-based measures should converge in the limit to 
the market-based valuation, thus providing a basis for testing the validity of using the 
internal efficiencies to derive a “synthetic price” for the value of a Firm (Hayek, 945, 
September, p. 35). 

In fact, the latter should reflect more accurately the “true” value of a Firm, which is 
what “Value Investors” and Hedge Funds are constantly trying to identify. Indeed, 
perturbation-response models offer a means to quantitatively link micro, Firm-level models 
to macro, policy-level models. 

Pricing Public-sector Investments 

Measures of investment efficiency for the public sector enable a portfolio manager to 
keep investments allocated to the most profitable outcomes—despite shocks and 
perturbations to operations. But, effective Portfolio Management depends on accurate 
pricing information. When information is limited or uncertain, risk is not efficiently priced. As 
a result, the marginal social utility of an investment will not equate to its price—thus, leaving 
no means to identify a “socially optimal” level of investment in either the private or the public 
sector. The net effect is analogous to Nash Equilibrium: a sup-optimal investment 
equilibrium condition that occurs when no player has incentive to unilaterally change 
strategy—because a change by any one of them would lead that player to earn less than 
would be obtained by remaining with the current strategy (Cover & Thomas, 1991, p. 460, 
475; Dixit & Pindyck, 1994, p. 147, 283). 

Sub-optimality can be driven by limited information, by a lack of incentives to change 
(as is the case for the Nash Equilibrium), as well as by uncertainty driven by factors such as 
market volatility. Higher volatility estimates reflect greater-than-expected fluctuations in 
underlying price levels and result in higher-option premiums for both puts and calls. With 
respect to the internal processing efficiencies of a Firm, the Taguchi’s common and special 
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sources of variability drive the perturbations of a Firm’s workflow (Leon, Shoemaker, & 
Kacker, 1987). The efficiency of response determines whether and to what extent a Firm 
can synthesize information into actionable decisions in a timely manner. The often 
inconclusive, interwoven, and ambiguous nature of available information can result in time 
consumed to assess its value, which depreciates the time value of information. The resulting 
delay propagates uncertainty, which equates to opportunities lost. How much time is 
required to resolve the ambiguity is a function of a Firm’s ability to manage that variability, 
which depends on its ability to synthesize information. 

The impact of information efficiency can be measured in terms of response to 
random perturbations. The parameters characterizing the perturbations include the 
amplitude and time lag for changes in valuation and resource allocations, the amount of 
work/rework completed, requirements churn, the ratio of assets invested to return on those 
assets, output price/cost, fluctuations in labor, resource availability, the time to close major 
issues, the time to identify and to minimize both special and common causes of variability, 
schedule, and price, etc. Note that while these parameters indicate a Firm’s internal 
information processing efficiencies with respect to work progress, much as Earned Value 
does, they say nothing of business value—i.e., whether the product provides anything useful 
to a customer. 

The perturbation-responses are governed by structure of a Firm, which can be 
represented as a quasi-deterministic State Variable model. The combination of these 
techniques also can be used to identify the boundaries beyond which the magnitude of the 
perturbations would result in unstable responses and project failure (Ford & Taylor, 2006, 
pp. 337-369). 1 Information Theory benefits the analysis through the provision of algorithms 
that reduce the number of (perturbation-response) states to be considered in evaluating a 
process. The larger the number of states, the greater the lack of specificity or uncertainty 
(entropy) of the system. The following, highly over-simplified example illustrates the point. 

The number of states that a single dice can assume is 6. But, if we know that the 
dice is in some sense biased toward either even or odd numbers (i.e., we have additional or 
“side” information), then the number of states that need to be considered is effectively 
reduced to 3, thus changing the outcome probabilities but not the probability distribution 
model. Alternatively stated, the entropy (disorganization) of a system does not depend on 
the actual values taken by the random variables describing it, but only on the associated 
probabilities. 

Changes in the probability space, driven by changes to the efficiency of information 
flows within a Firm, provide a quantitative basis for linking the efficiency of 
information/knowledge management to well-defined mathematical processes, thus providing 
quantitative measures of risk that correspond to the underlying physical processes. 

Information Theory can enable decision-makers to reduce risk, often drastically, by 
providing a quantitative framework to address issues such as: 

1. The quantitative determination of changes in the uncertainty levels associated with 

cost/schedule/resource estimates as a project proceeds through its lifecycle. 


1 Ford and Taylor’s text provides detailed discussion of project/program stability. 
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2. Discounting the anticipated benefits from a project—by measuring risk as a function 
of the time lags and amplitude of a Firm’s response to various types of perturbations. 

3. Identifying and reducing (both normal and special) sources of variability that 
adversely impact work progress and service/product quality 

4. Constructing rules to achieve optimal investments—in terms of ensuring that a Firm’s 
decision-making/information is efficient 

5. The use-state variable models to estimate whether, and at what rate, investment 
management is improving 

Models capable of answering these questions we consider next. 

Computational Models 

The Firm’s responses to the perturbations provide the raw data from which (indirect) 
measures of that efficiency can be expressed in terms of factors such as the time lag and 
amplitude of responses, and the variability of both. The perturbation/response processes are 
modeled using state variable regulator/ controller design methods. One of the best known 
and widely applied is the Kalman Filter, which can be used to measure how uncertainty 
propagates over time, and, thus, calculate the information-carrying capacity of a Firm. 

The output of the Kalman Filter model is the amount and rate of information gain 
produced by various organizational structures. The magnitude and rate of correction serve 
as measures of a Firm’s information processing capability, and (by implication) its level of 
entropy or level of internal-information organizational efficiency. The more efficient the Firm, 
the more quickly it will respond to perturbations (random shocks) regardless of source, 
internal or external. (The situation is analogous to determining the bandwidth of a 
communications system). 

Schematically, the Kalman Filter information flow/computation cycle is illustrated in 
Figure 1, below. 
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Kalman Filter 


Z~(k+1) 


x A (k+1|k+1) 



Legend: 

x is the vector of variables comprising the system, whose state is to be estimated over 
the successive time (e.g., multi-stage investment) periods k =0,1,2,... 

x A (k+1 |k) is the predicted estimate of x for time “k+1” based on measures taken at time 
“k.” 

z is the actual, uncorrected measurement of x. 

z ~ is estimate of x when corrected errors are introduced by the measurement process, 
z A is the estimate of x A as filtered by H. 

H is the measurement transformation matrix that relates the system state vector, x, to its 
measure, z. 

Delay is the lag in system response to a stimulus. Delay is inherent to an 
organization because information cannot be gathered, analyzed, or transmitted 
instantaneously. Thus, changes in the environment or slips in schedule may or may not be 
recognized when they occur. For example, decreases in data quality typically generate 
increased disruption in operation. As more resources are shifted to fixing and correcting 
data records, the rate at which information is processed decreases. The resulting 
inefficiency generates increased correction and rework rates, along with increase delays in 
task completion. 
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F is the coefficient matrix of the state variable vector x. It describes the input/output 
efficiencies of a Firm and its investments. If these change overtime, then F (k) =/= F 
(k+1), k=0, 1,2... and the system described by Eqn (1), (2), below, would be non-linear 
in time “t.” 

The steps presented in Figure 1 are as follows: 

1. Begin with x A (0 |0) =0, for k = 0, 1,2... Use x A (k | k) to iteratively compute 
x A (k +1 ,k +1) given z(k+1). 


Where: 


x A ( k | k) Is the estimate propagated forward by pre-multiplying it by F(k+1, k) to 
give the predicted estimate x A (k+1 |k). 

2. Pre-multiply x A (k+1 | k) by H (m +1) to compute is z A (k+1 | k), which is then 
subtracted from the actual measurement z (k+1). 

■ The result is z~(k+1 | k), which is the measurement residual—that is, the 
difference between actual and predicted estimates at time (k+1). 

3. The residual, z ~, is pre-multiplied by K (k+1), the Kalman Filter coefficient matrix, 
which is added to x A (k+1 | k). 

■ The result is x A (k+1 | k +1), which is the estimate of x at time (k+1), given the 
measures updated at time (k+1). 

4. x A (k+1 | k +1), the optimal filtered estimate, is stored until the next measurement is 
made—at which time the cycle is repeated. (This is the only data that needs to be 
stored between measures, thus saving considerable computer storage and memory.) 

Note that the product K (k+1)*z~(k+1 | k) is the correction that is added to the 
predicted estimate x A (k+1 |k) to determine the filtered estimate. 

The optimal filter consists of the model of the dynamic process which performs the 
function of a prediction and feedback correction scheme in which the gain-times-residual, K 
(k+1) *z~(k+1 |k], per Eqn [3] below enters the model as the forcing function u( k) (Meditch, 
1969, p.182). 

How all this works is illustrated by the following state-variable model for which the 
following assumptions are made. 

■ Time delay decreases the volume of work accomplished per unit of time. 

■ Time delay is driven by unstable/poorly managed requirements, funding 
instability, etc., thus acting to increase the amount of rework. 

■ In this example, model parameters are assumed to be Gaussian (Normally) 
Distributed, with side information (e.g., feedback) entering into the system as a 
sequence of predict-correct actions. 

x(t) is the vector of state variables, which consists of the two elements: 
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x-t (t) =Volume of work/unit of time, 
x 2 (t) = Volume of rework/unit of time. 
The state variable model is: 


[ 1 ] x’ = F* x (t) + u(t) 

(where x’ = dx (t)/dt) = [x-,’, x 2 ’] = [dXi (t)/dt, x 2 ’ = dx 2 (t)/dt] 

[2] [ Xl ’]= [1 1] [ Xl (t)] +[1 0][ Ul (t)] 

[x 2 ] [0 1] [x 2 (t)] [0 1][u 2 (t)] 

Verbally, Eqn [2] states that the rate of change in work package completion is the 
sum of the current workload (xl) + rework (x2) + the arrival of new work (ul). The rework 
rate (x2’) is equal to the sum of current rework (x2) + new rework (u2), where the vector u(t) 
is governed by the corrections provided by Eqn [3], 

The control vector u(t) = [u^t), u 2 (t) ] is defined in terms of predictor/corrector 
parameters as: 

[3] u(t) = K(t)*[z - H* x A ] 

Eqn [3] computes the correction, given x A from the Kalman Filter Eqn [9], below. 

[4] e(t) = K (t)*[ v - H *( x - x P )] 

Eqn [4] computes the error estimate prior to measurement. 

Eqns [3], [4] derive from the linear measurement Model: 

[5] z = H* x + B * v 

To keep the computation simple, H, B are defined as identity matrices: 

H = [1 0] 

[0 1] 

B = [1 0] 

[0 1] 

v is a vector of random measurement errors which are independent of the state x. (v 
is implemented with Monte Carlo simulation input.) 

Eqn [1] and Eqn [2] can be cast into Kalman Filter format by defining the following 
parameters: 

X P is the Estimate of the system state prior to measurement, as defined in [7-P], 
below. 
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P P is Covariance of the system state prior to measurement, as defined in [7-P], 
below. 

x A is the Estimate of the system state after measurement, as defined in [7-A], 

P A is Covariance of the system state after measurement, as defined in [7-A], 

[6] R = E[v*v’] Is the mean of the measurement error vector. 

[7-P] P p = E[(x p - x)*( x p - x) T ] Is the error covariance matrix prior to 
measurement. 

[7-A] P a = E [(x a - x)*(x a - x) T ] Is the error covariance matrix after measurement. 

[8] x a = x p + P p *H p T *R p _1 (z - H* x p) is the optimal estimate. 

Eqn [8] is a model of Eqn [1] with a correction term that is proportional to the 
difference between the actual measurement z-, and the predicted measurement H*xj 

To minimize the subscript clutter used to denote the before and after calculations, P A 
is changed to P, for after, and P p to M, for prior. Thus, indexing Eqn [8] for measures over 
successive discrete time periods becomes Eqn [8’]. 

[8’] x A i = x " +K j (z j - H * xi'), where i =m, m+1 .... 
for multi-stage investments. 

The Gain Matrix for the Kalman Filter, for time periods i = 1, 2... is 

[9] K j = P j * H i T *R j _1 

Eqn [9] can be interpreted as the proportionality matrix or the ratio between 

uncertainty in the covariance matrix P j , after measurement at time “i” and uncertainty in the 

measurements R j Eqn [6] (which can be interpreted as the effectiveness of a management 
reporting system). 

The propagation of uncertainty in the discrete time system, Eqn [11], below, is based 
on the computation of: 

[10] Pj = (M j' 1 +H j t *R j _1 *H j)' 1 

[10’] Pj = Mj-Mj*Hj T *(Hj*Mj*Hj T + R j)' 1 H j * M j 

[11] M j +1 = F j *P j *F j T + G j *Q j *G j T 

Eqn [11] reflects the balance between the new information-forcing function, 
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G j *Q j *G j 1 ; and, information processing efficiency, F j *P j *F j 1 . (In a traditional 

physical system, F j would represent the damping efficiency of the system. 
Collectively, Eqn [10], [10’], and [11] describe the propagation of the covariance of 

the error estimate, which are independent of the measurements z j. 

Eqn [9]-[11] Is the Kalman Filter for a multi-stage process. 

Prediction beyond the last measurement for states indexed as i = m +1, m+2, ... is 
given by Eqn [12], 


[12] x (j +1 ) - F (j) * x (i) + G (j) *U (j) 


Figure 2, below, provides a heuristic illustration (via the use of “canned” data) of the 
damping out effects of the Kalman Filter (illustrated by the magenta colored line), the 
retardation of that effect induced by response delay (as denoted by the green dashed line), 
with the black line illustrating the impact of a special cause of variability, such as the failure 
of an integration test, a reduction in funding, a major equipment failure, etc. The blue 
dashed line (highly exaggerated) illustrates “normal” variability causes, which might include 
ambiguous governance, the impact on products and services due to aging equipment, but 
which shows gradual improvement overtime. 



Figure 2. Perturbation-response Output from Kalman Filter 
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Summary/Next Research Steps 

The strategy and models outlined in this brief paper indicate how Information Theory can 
be used to quantify the value of information and translate that value into a “price” for public- 
sector investments that, in principle, is comparable in a competitive market. 

The next steps include: 

■ Fully defining the conditions under which the synthetic prices for public-sector goods are 
comparable in competitive markets. 

■ Developing methods to normalize comparisons across portfolios of diverse investment 
projects. 

■ Applying and evaluating the models using “real” data from public- and private-sector IT 
infrastructure investment projects. 

■ Using the model output to evaluate “synthetic prices” based on information gains defined 
using Kalman Filters. 

■ Comparing model results across a range of scenarios. 

■ Identifying algorithm improvements that accelerate convergence to specific price 
solutions. 

Glossary 


Term 

Description 

Black-Scholes model 

A mathematical model used to calculate the value of a project 
or an investment as derived from market-place dynamics, 
based on approximations to Brownian motion processes 

Brownian Motion 

The random motion of particles in a liquid. The mathematical 
model of describing this motion is the Wiener process. A 
continuous time process that forms the basis of many 
important mathematical models in thermodynamics and 

Finance 

Binomial Lattice Methods 

An algorithm for valuing complex option problems whose 
payoff depends on multiple state variables following correlated 
geometric Brownian processes 

Capital Asset Pricing (CAP) 

The concept that there is a true value for a stock 
corresponding to its risk, for which various computational 
models can be used to determine risk-adjusted discount rates 
for investments, and to decide whether a stock price is too 
high or too low. 

Complexity 

There are at least two basic types of complexity: 

Descriptive complexity of an object—Kolmogorov complexity. 
While not directly computable, it can be bound between 
computable measures to describe the complexity of a 
sequence of symbols; Computational complexity—measures 
the time or space required for a computation 

Derivative 

A financial instrument that derives its value from the value of 
some other financial instrument or variable. For example, a 
stock option is a derivative because it derives its value from 
the value of a stock. An interest-rate swap is a derivative 
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because it derives its value from one or more interest rate 
indices 

Dynamic Hedging 

The purchasing/selling of financial instruments to reduce or 
cancel out the risk in another investment as required by 
changing market conditions 

Earned Value 

Measures the dollar-value work completed per unit of time. It 
is a measure of progress against an objective, from which 
schedule (SC) and cost variances (CV) can be computed 
using Planned Value (PV) and Actual Cost (AC). 

SV = EV - PV 

CV = EV-AC. 

But, realistic estimates of Planned Value are seldom 
available—especially at the on-set of a large, complex project. 
Hence, the value of Information Theory lies in determining 
when and to what extent confidence can be placed in a 
benchmark such as Planned Value 

Efficient Frontier 

A concept that there is a true value for a stock corresponding 
to its risk; this theory of stock price is called Capital Assets 
Pricing Model and is used to decide whether a stock price is 
too high or too low. 

Entropy 

A measure of the disorganization of a physical system/The 
uncertainty of a single random variable/The minimum 
descriptive complexity of a random variable. An irreducible 
level of complexity below which a signal cannot be 
compressed 

Equity Markets 

A (competitive) stock market that efficiently coordinates 
decisions involving time and uncertainty 

Exogenous 

Refers to variables whose values are driven by factors 
external to the firm, or processes, of interest 

Firm 

In this paper, the “Firm” refers to an organization in either the 
private or public sector tasked with investing in and developing 
new products and services 

Game Theory 

The branch of applied mathematics and economics that 
studies situations where players choose different actions in an 
attempt to maximize their returns. It provides a formal, 
quantitative modeling approach to social situations in which 
decision-makers interact 

Information Theory 

A discipline spanninq mathematics, economics, phvsics, 
communication theory, statistics, involving the quantification of 
data. For communications, the goal is to enable as much data 
as possible to be reliably stored or communicated over a 
channel. 

Ito’s Lemma 

Is used to integrate and differentiate stochastic processes. An 
/to process can represent the dynamics of the value of a 
project which does not have a time derivative in the 
conventional sense- because its fluctuations over (short) 
periods of time do not have derivatives 

Kalman Filter 

A mathematical algorithm that operates in a predict-correct 
fashion that uses feedback to maintain a system (e.g., a 
rocket, or an investment portfolio) on a desired trajectory 

Levy Process 

A generalization of Brownian Motion processes to include 
discrete state jumps. The jumps can be local, global, 
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simultaneous, independent or correlated 

Law of Large Numbers 

The sum of independent, identically distributed random 
variables that can be approximated arbitrarily closely to the 
expected value of the random variables 

Markov Property 

The property of a process that current information is useful for 
forecasting the future path of a process. Applied to Stock 
processes on the premise that public information is quickly 
incorporated into the current price /D-63/ 

Mutual information 

The communication rate (efficiency) in the presence of noise. 

It is a measure of the amount of information that one random 
variable contains about another random variable. It is the 
reduction in the uncertainty of one random variable induced by 
knowledge of the other 

Nash Equilibrium 

A condition in Game Theory in which no player has incentive 
to unilaterally change her action. Players are in equilibrium if a 
change in strategies by any one of them would lead that 
player to earn less than if she remained with her current 
strategy 

Net Present Value (NPV) 

NPV is a standard method for the financial evaluation of a 
lonq-term project. Used for capital budqetinq, and widely 
throuqhout economics, it measures the excess or shortfall of 
cash flows (in present value (PV) terms) once financinq 
charges are met. By definition, NPV = Present value of net 
cash flows 

Options Pricing 

Is a contract between a buyer and a seller, or a provision of a 
contract, that gives one party (the option holder) the right, but 
not the obligation, to perform a specified transaction with 
another party (the option issuer or option writer) according to 
specified terms. Option contracts are a form of derivative 
instrument 

Portfolio Management 

The discipline of managing a portfolio of investments with the 
objective of maximizing the value of the entire portfolio by 
reallocating resources among the investments comprising the 
portfolio 

Public-sector Firms 

Firms such as public health or security tasked with providing 
goods and services whose valuations are not saleable in 
private equity (i.e., stock) markets, but which benefit society 

Random Walk 

A process that takes a discrete move in a specific direction 
according to a specified probability distribution. The Brownian 
Motion (Weiner process) is the limit for the discrete random 
walk process 

Rate Distortion Theory 

A maior branch of information theory; it addresses the problem 
of determininq the minimal amount of entropv (or information) 
that can be communicated over a channel, so that the source 
(input signal) can be approximately reconstructed at the 
receiver (output signal) without exceeding a given distortion 
level. 

Risk 

“Risk" is randomness with knowable probabilities; 

"uncertainty" is randomness with unknowable probabilities. 
Frank Kniqht (1921): An enqineerinq definition of risk is “the 
(probability of an adverse event) x (loss per event)” 

ROI—Return on Investment 

A measure of the net income a firm is able to earn with its total 
assets. Return on investment is calculated by dividing net 
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profits after taxes by total assets. The Rate of Return (ROR) 
or Return on Investment (ROI), or sometimes just return, is the 
ratio of money qained or lost on an investment relative to the 
amount of money invested. The amount of money gained or 
lost mav be referred to as interest, profit/loss, qain/loss, or net 
income/loss. The money invested may be referred to as the 
asset, capital, Drincioal, or the cost basis of the investment. 

Side information 

Information that is relevant to the outcome of an event such as 
a coin toss, or a horse race 

System Control Theory 

The discipline of controlling complex machines such as 
aircraft, computer networks, financial and manufacturing 
systems 

Uncertainty 

A characteristic of a random variable that is measured as 
Entropy. It is the number of bits required to describe a 

Random Variable. The larger the number of values that the 
random variable can take, the larger the uncertainty. 

Variability: 

Special and common sources 

A common source of variability: The quality level of an item 
created by a machine tool that is wearing out. 

A special source: an unlikely event such as the breaking of an 
artifact being processed by machine tool that fails 
unexpectedly 
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2003 - 2006 Sponsored Acquisition Research Topics 

Acquisition Management 

■ Software Requirements for OA 

■ Managing Services Supply Chain 

■ Acquiring Combat Capability via Public-Private Partnerships (PPPs) 

■ Knowledge Value Added (KVA) + Real Options (RO) Applied to Shipyard 
Planning Processes 

■ Portfolio Optimization via KVA + RO 

■ MOSA Contracting Implications 

■ Strategy for Defense Acquisition Research 

■ Spiral Development 

■ BCA: Contractor vs. Organic Growth 
Contract Management 

■ USAF IT Commodity Council 

■ Contractors in 21st Century Combat Zone 

■ Joint Contingency Contracting 

■ Navy Contract Writing Guide 

■ Commodity Sourcing Strategies 

■ Past Performance in Source Selection 

■ USMC Contingency Contracting 

■ Transforming DoD Contract Closeout 

■ Model for Optimizing Contingency Contracting Planning and Execution 
Financial Management 

■ PPPs and Government Financing 

■ Energy Saving Contracts/DoD Mobile Assets 

■ Capital Budgeting for DoD 

■ Financing DoD Budget via PPPs 

■ ROI of Information Warfare Systems 

■ Acquisitions via leasing: MPS case 

■ Special Termination Liability in MDAPs 
Logistics Management 

■ R-TOC Aegis Microwave Power Tubes 

■ Privatization-NOSL/NAWCI 

■ Army LOG MOD 
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PBL (4) 

Contractors Supporting Military Operations 
RFID (4) 

Strategic Sourcing 

ASDS Product Support Analysis 

Analysis of LAV Depot Maintenance 

Diffusion/Variability on Vendor Performance Evaluation 

Optimizing CIWS Lifecycle Support (LCS) 


Program Management 


Building Collaborative Capacity 

Knowledge, Responsibilities and Decision Rights in MDAPs 
KVA Applied to Aegis and SSDS 

Business Process Reengineering (BPR) for LCS Mission Module Acquisition 

Terminating Your Own Program 

Collaborative IT Tools Leveraging Competence 


A complete listing and electronic copies of published research within the Acquisition 
Research Program are available on our website: www.acquisitionresearch.org 
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Managing Rncsrtiinty find Risk in 
Public lector Investment© 


R. Suter, RS Consulting 
( , May 17,2005) 



Managing DoD IT Infrastructur® 

Investments 


The DoD annual budget approaches a half trillion 
dollars 

► Much of that budget is invested either directly or 
indirectly affected by Information Technology 
Infrastructure modernization initiatives 

► Because there is no market for public sector goods, 
efficient resource utilization is a challenge 



-oit/-omplexity Clivers 


Complexity increases exponentially with 
size, rate and scope of technology 
innovation 


Cost of integration is driven by the 
exceptional complexity of IT infrastructure 


^DoD is exploring the building of system 
of-systems, and federation of systems 



^“ensures o E^ectivenese or ^ublic 

lector Investment 

► Do not capture underlying process dynamics 

► Are hard to apply 

► Tend to be not timely - Are lagging indicators 

► Are vulnerable to “gaming” 



Impiet to ^ublfc ©ustor lnv©stm®nt 


►Systemic under estimation of cost and schedule 
at the onset of major systems/ software programs 

►Accountability difficult to establish 


►Often deliver less value than promised 



Investment in the ^ublie Sector 


Unlike the Private Sector, Public Sector markets 
lacks mechanisms for: 

► Self-regulation 

► Rapid knowledge dissemination and aggregation 

► “Inherent” incentives linking expenditures and 
and accountability 




Pricing with Market© 


► Irrthe Private Sector risk and reward are set by 
Competitive Markets 

► The price of a stock represents the market 
consensus of the value of a Firm 

- Which may/may not be accurate 

► Value depends on delivering the right product 
efficiently 



Pricing Without Market® 


■ 


► No clear way to link price and value 

► But the price of a Public Sector firm ultimately depends on 
its internal efficiency 

► In theory, internal efficiency based valuation and and market 
valuations should converge to a single value 

► Internal efficiencies are measurable 

S Regardless of whether the Firm is in Private or Public 
Sector 




Whiit Competitive Merkete Co 


► Enable a rapid consensus of a Firm’s worth 

■ Its value - its measure is the price 

►The factors governing that consensus are: 


^The rate at which information becomes known 


^The rate at which price/risk information is 
aggregated (e.g., decision formation) 



The Basis of p rce Formation 


► Knowledge Diffusion Rate 

^Tends to keep rule violations in-check 


► Information Aggregation Rate 

^Bounds the timeliness & quality of 
investment decisions 



Information ^iffution and 
Aggregation Rates 


► Determine whether, and at what rate, price 
consensus occurs 


Indicate market efficiency 

Y Sub-optimal rates result in price discrepancies 
that can be exploited by floor traders, value 
investors, and others 



nctrtalnty, ^iuk and nv®@tmant 

Valuation 


► In tial project cost/schedules estimates are subject 
to significant uncertainty 

► The ability to reduce uncertainty and risk depends 
significantly on a Firm’s internal efficiencies 

► That ability is a consequence of the Firm’s internal 
efficiencies 

s Its measure is a “synthetic price” 



nvestment isk driver© 
Software intensive system© 


Three primary risk drivers: 

►Technical complexity 
- ► Syst em Integration complexity 


► Project size and duration 


These risks govern uncertainty, irreversibility, and 
timing for IT infrastructure investments in both the 
Private and Public Sectors 



In restructure Investment Parameters 


► Uncertainty 

^Software intensive systems are particularly 
sensitive to the under-estimation of risk 


> * The l eve l of co mplexity is hard to comprehend 
let alone manage or measure 


►Timing 

^Technology investments can be rendered 
obsolete by: 


■New technology 


■Evolving threats 



I investment Management Parameters 

► Irreversibility 

■ Investment is allocated to the labor required to develop 
the intellectual capital embedded in software is 
unrecoverable 

^Zero salvage value unless deployed 


■ Large scale IT infra-structure investments are 
particularly susceptible to these parameters 


Mastery of internal operating efficiencies is crucial 



Spao'al and Common Causes o Variability 

► Perturbations in a Firm’s performance derive from 
twp sources of variability: 

► Common causes : Poorly defined requirements, 
processes, procedures, aging equipment 

^These are entirely controllable 

s Are the focus of 6-Sigma, the CMMI 

► Special causes : equipment failures, spikes in staff 
turnover, unrecognized sources of integration 
complexity, etc. 



he Impact of Variability 


► A Firm’s performance, and risk, is governed by the 
ability to master these two primary causes of 
variability 

► Less efficient Firms will exhibit growing variability 
overtime/declining performance, regardless of 
sector: Public or Private 

► Variability propagates uncertainty thus driving risk 
via delays in decision making 

^The time value of available information degrades at an 
accelerates rate 




Variability and Response Perturbations 


► Variability drives sy stem perturbations 
vwhich are stochastic 


► Perturbation responses indicate the efficiency of 
underlying of information diffusion & aggregation 
rates 


► The rates are not directly observable 
VBut govern “synthetic” price formation 



The Bas s for Synthetic Prices 

► A form is a stocha stic feedback system that can be 
modeled using algorithms from System Control and 
Information Theory 


► The efficiencies of information utilization are 

quantifiable as “Information Gains” - derived from a 
Kalman Filter 


► But, producing information gains: 

v' Consumes resources (time, labor, technology) 
^Produces benefits (e.g., on-time product/service delivery) 



Th® Relation between Synthetic Price 
and Black-Scholes Model® 


Common Objective: 


Both determine the “worth” of an investment as measured 
against various combinations of risk, uncertainty, interest 
t; fltes, and competing i nvestment opportunities 


vf 


Neither predict which Firms are most likely to succeed 


Both Synthetic Price and Black-Scholes models: 

► Are based on the Law of Large Numbers 

► Converge, in the limit, to “true” market based valuations 



Capital Prfcing 
he Blaek-lsholet Model 


Black-Scholes assumes that stocks have a true 
value that: 

f Corresponds to its risk 

Determines whether the market price for a 
stock is too high or tool low 


stock option’s value equals the value of the 
information concerning that risk 



Signiff csnc© of ilack-Scholt© 

►Stock Value dynamics are modeled as Brownian Motion 
process 


►Captures market dynamics in terms of a few variables 


+ 


► Provides a (real time) computationally efficient models 
that link price, interest, and discount rates 


► Eschews unobservable/hard to measure parameters such 
as “Investor Psychology” 


^Demonstrates that effective decision making need not 
depend on a detailed understanding of causality 



3ynthet c Pricing - 
Meatunng Information Gains 


Apply Kalman Filter models to measure: 


► How uncertainty propagates over time 


►The information carrying capacity/efficiency of 
Firm 


^And thus its value 

Synthetic prices are measurable as a “gain” in a 
Kalman Filter 



Kalman Filter Schematic 





x A (k+1|k+1) 





Oytput from Kalman Filter Modal 
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Legend 

u^t) - input forcing function -special variability cause 
u 2 (t) - input - common variability cause 

Are damped out by steady process improvement 
P A (t-d 2 ) - Error propagation (variability) driven by “long” delay 

P A (t- d.,)- Impact of “short” delay on system ability to minimize error propagation for (d.,< d 2 ) 











N ext Steps 

► Va li dat e that P ertufbation based measures 
converge, in the limit, to the market based valuations 


►Get data for several hundred public & private sector 
Firms; 

^Normalize the data across the Public and Private 
sector 

►Quantitatively estimate efficiency and information 
gain parameters 


►Validate models against the predictions they make 



Backup 3lfd©s 




Computation Notes 


x*a vector of variables comprising the system, whose 
state is to be estimated over the successive time (e.g., 
multi-stage investment) periods k =0,1,2,... 


x A (k+1 |k): The predicted estimate of x for time “k+1 ” 
based on measures taken at time “k” 


z: the actual, uncorrected measurement of x 

z ~:The estimate of x when corrected errors introduced by 
the measurement process 

z A : The estimate of x A as filtered by H 

H: The measurement transformation matrix that relates 
the system state vector, x, to its measure, z 




Company 

Size 

Success 

Rate '94 

Success 
Rate '98 

Large 

9% 

24% 

Medium 

16% 

28% 

Small 

28% 

32% 





Turning Chaos into Success Jim Johnson 


Project 

Cost '94 

True Cost 
'98 

Delta 

$2.3M 

$1.2M 

-65% 

$1.3M 

$1.2M 

-41% 

$.4M 

$1.1M 

-4% 


, 12/99 




























'to 7 ® Lemma 


• Use to analyze processes without time derivatives, 
including output prices, input costs, etc... that cannot 
be manipulated using the ordinary rules of calculus 

• The fundamental Theorem of Stochastic Calculus- 
use to analyze infrequent, discrete jumps 

• Kolmogorov equation describe the dynamics of the 
pdf for a stochastic process 



Problem Dimensionality 


• Model Time frames: short/long term 

• Sources of variability: common/special 

• Levels of analysis: Micro/macro 

• Uncertainty and Risk 

• Forcing functions: stochastic 

• Response functions: profit driven/”inspection 
driven 

• System Stability range... 

• Transient/Steady state response 






Estimate variability 


Cost Estimate Uncertainty Reduction as a Function of Project Life 

Cycle & Process Maturity Level 


Estimate Variability Envelope 


1st NASA Estimate/ 
logical design 
Milestone 3 

NASA Preliminary . 

Estimate at conceptual design 


3rd NASA 
Estimate/ 

1 st release 
Milestone 4 


+ 75% 


Detailed 
Design/ 
2nd NASA 
Estimate 


NASA 40% Estimate Growth Line for Cost 


System concept 


Detailed Requirements 


Architecture 


Integration/Test 


Approval 
Milestone 1 


Milestone 2 


Deployment 
Milestone 5 


Finish 


Project Life Cycle -> 


Figure 3: CMM Level 3 Projects at NASA - Software Engineering Laboratory (SEL) 
(http://www.nasa.goddard.gov) 






